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Device Breakdown 


• 6 consoles (Xbox 360, PS3, WiiU, Xbox One, PS4, and Apple TV) 

• 9 tablets (iPad mini and beyond) 

• 7 phones (iPhone 5 and beyond) 




GAME DEVELOPERS CONFERENCE March 14-18, 2016 - Expo: March 16-18, 2016 #GDC16 


GDC 


Main Challenges 

• Memory 

. 512MB to 8GB 

• Computing Power 

• 2 cores to 8 cores 

• Verification 

• 300+ characters, story levels, racing, online, and 


more... 
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Other Challenges 

• Input 

• Gamepad, touch screen, remote control, etc... 

• Media 

• DVD, Blu-ray, Solid State, etc... 

• 1 st Party Requirements 

• Disc eject, loss of network, iconography, etc... 


GDC 
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Talk Overview 

• Goals 

• Scalable Technology 

• Working with Budgets 

• Verification 

• Teams 

• Postmortem 
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Experience Goals 

• Core experience 
on all devices 

• Expectations met 
on all devices 



GDC 




GAME DEVELOPERS CONFERENCE March 14-18, 2016 • Expo: Marc 


Developer Goals 

• Minimize custom 
work 


Content creators 
control quality 
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What is Scalable Technology? 


• Easy 

• Invisible 

• Directed 


-,J' - Vicarious Visions 

ALCHEMY 
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Device Classes 


• Gen8 Console 

• Xbox One and PS4 

• Gen7 Console 

. WiiU, Xbox 360, and PS3 

• iOS High 

• Apple TV, iPad Air+, iPad mini 2+, and iPhone 5s+ 

• iOS Low 

• iPad mini+ 



Emulation: Gen8 Console 
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Types of Technology 

• Automatic 

• 80 % of the way 

• Can opt out 

• Directed 

• Improve quality using context 


GAME DEVELOPERS CONFERENCE March 14-18, 2016 - Expo: March 16-18, 2016 #GDC16 


GDC 


Automatic: Rendering Features 



Render 

Style 

Dynamic 

Lighting & Shadows 

Materials 

Features 

iOS 

Low 

Forward 

None 

Simplified 

iOS 

High 

Forward 

Partial 

Standard 

Gen7 

Deferred 

Standard 

Standard 


Post Processing 

Anti-Aliasing 

Low 

None 

Low 

MSAA 

Medium 

FXAA 

Medium 

High 

FXAA 

High 


Gen8 


Deferred 


Advanced 


Advanced 
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Automatic: Texture Resampling 

• Target 1080p 

• Exclude 16k pixels 

• Could opt-out 



Clamp 

3D/VFX 

UI 

iOS Low 

1024x1024 

75% 

50% 

iOS High 

1024x1024 

50% 

0% 

Gen7 

1024x1024 

50% 

33% 

Gen8 

2048x2048 

0% 

0% 
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Automatic: Geometry Decimation 


• Error tolerance 

• Less on characters 


Error 

Tolerance 









Gen 8 



bis 

ite 
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iOS Low 



Gen 8 



iOS Low 





GDC 




GAME DEVELOPERS CONFERENCE March 14-18, 2016 • Expo: March 16-18, 2016 #GOC16 



Geometry Decimation: Opt Out 
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Automatic: Audio Features 



iOS Low 
iOS High 
Gen7 
Gen8 


Reverb 

Flanger 

Equalizer 

Compressor 

Echo 

None 

None 

Simplified 

Simplified 

None 

None 

None 

Standard 

Standard 

Standard 

Advanced 

Advanced 

Advanced 

Advanced 

Standard 

Advanced 

Advanced 

Advanced 

Advanced 

Standard 


Sample 

Rate 

Clamp 


24khz 

24khz 

None 

None 


*Also supported per device options 
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Directed Scalability 

• Toggles 

• X thing on Y platform 

• Importance 

• Cull low priority 
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Directed: Entity Layers 


i A 








lvl Outliner lvl Object Library 


Search Filter... 


o # 


Object 

ly >e 

O «, 

B > 

Areal_Npcs* 

Layer 

O - 


Areal_N p c s_N oT a b 1 et 

Layer 

O * 


CGameE ntityOG2 

CGameE itity 

O 


CGameE ntityGQS 

CGarmeE itity 

O 


CGameE ntityGQS 

CGameE itity 

O ■*. 


CGameE ntityOQ9 

CGameEntity 

o 


CGameEntityOlO 

CGameEntity 

o * 


CGameEntityOll 

CGameE ntity 

o * 


CGameEntityG13 

CGameEntity 

o * 


CGameE ntity014 

CGameEntity 

o *■ 


CGameE ntity015 

CGameEntity 

O ‘ 

@ > 

Areal_Static 

Layer 

o ■* 

@ > 

Area2_Cameras 

Layer 

o * 

@ l> 

Area2_Collectibles 

Layer 

o 


Area2 DriverZone 

Laver 

1 object selected 


M A 

High Tablet Disabled 




Gen 8 



iOS 
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Directed: VFX Layers 



Gen 8 


iOS 
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Directed: Visual Scripting Node 


Platform Switch 


¥ 

In Gen 8Console 

¥ 


Gen 7Console 

¥ 


Tablet High 

¥ 


Tablet Low 

¥ 


■ 










General: Book Enter & Exit 



On Enter 


Platform Switch 


Gen SConsole 


On Enter 


Check Gate 


Gen 7Console 
Tablet High 
Tablet Low 


Target Times To Fire 


Auto Close Count 


Spawn Effect On Entr 


Target Source Spawned Effect 


Owner 


LetterBox VFX 
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Directed: Importance Authoring 

• Assign importance values 

• Remove less valuable things 


D i rected :CI m po rta n ce 












Directed: I 


i 





mportance Visibility 



- - 


0 









r 

Directed: Importance Visibility 


« 


Directed: Importance 
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Good Content Budgets 

• Specific 

• Actionable 

• Realistic 

• Stable 

• Relative 
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Memory 

• 2 Types 

• Main memory 

•Scripts, UI, audio, etc... 

. VRAM 

•Models, textures, etc... 

• PS3 exceptions 

• SPU utilization 


2016 #GDC16 
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Main Memory Budgets 


• Similar across devices 

• Broke some rules 

• Not stable 

• Not actionable 

•Lots of assets in one pool 
•Validated at runtime 

• Required engineering 



ink 
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VRAM Budgets 

• Different per device 

• Followed most rules 

• Stable 

• Actionable 

•Verified during build 
•Good tools 

• Free of engineers (mostly) 




Budgets in MBs 









GDC 


<> 


GAME DEVELOPERS CONFERENCE March 14-18, 2016 • Expo: March 16-18, 2016 #GOC16 




Reports: Daily 


Level 

Free At Peak KB 

Increase KB 1 

Sky3 

856.42 

- 251.84 


974.38 

45.78 

z_Magiczone 

1066.90 

- 6.12 

Natl 

1074.78 

- 26.97 

Darkl 

1244.35 

- 81.31 


1253.71 

4.22 

Skyl 

1365.31 

- 47.59 

z_Lightzone 

1497.09 

34.92 

z_Earthzone 

1595.92 

- 360.23 


1684.99 

- 79.17 

BUZ 

1697.69 

- 1.69 

z_Airzone 

1820.79 

- 9.00 

Sky2 

1823.93 

- 77.36 

W55EL 

1846.96 

21.99 


1901.04 

25.14 

tsmsssm 

2155.59 

30.77 


Main Memor 


Levels 


Level 


Increase MB 

Remaining I 

Natl 

139.74 

- 0.09 

0.98 

Skyl 

137.80 

- 0.01 

2.92 


137.16 

- 0.67 

3.56 

Introl 

136.22 

- 0.32 

4.50 

Quest2 

133.90 

- 0.08 

6.82 

Finall 

130.63 

- 0.30 

10.09 

Quest3 

130.33 

- 0.16 

10.39 


126.97 

- 0.32 

3.75 

Nat2 

125.78 

- 0.08 

14.94 

Darkl 

120.43 

- 0.21 

20.29 

Nat3 

114.12 

- 0.16 

26.60 

Esra 

111.54 

0.00 

29.18 

Questl 

96.44 

- 0.19 

44.28 

IMH 

93.50 

- 2.28 

47.22 

Final2 

30.69 

0.00 

110.03 

Intro 

12.66 

0.00 

128.06 


VRAM 
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Reports: Memory vs Changelists 


3000 


2750 


2500 


2250 

2000 

1750 

1500 

1250 

1000 

750 

500 

250 






•OOOO* 


it 


W/ • 


it- to tc- in' h- ^ to co cj tc- trz- ct r^r ^ ^ ^ un- Ch;- ^ tc- irj ^ ^ ^ o ry cy 

Cd-K n. h- (fc&T- f^fyfyfyn-^f^r^rij^iJTiJT'fvfvcrfc^nT^r^r^rLritEK cc-cem3'tJ}(Jj(J3-0jCjr- CyCyfyCyCyiTTcrfcrfcri-^f-^r'T^rtoLrjiJTLni-iJTiJTCDtDtD-tDtDtE-K. iccD(JjOjfycr!irtr>cr}i7ilN. ro Gi <3> S> d r- crfCTMTiiTT^r'^r^r^r'T’TtoLn-LnjLrjLnjLni'tEi-CDtE-i 1 ^ K 
0 00&&T-T--^^7^^^^ T --,T--,^n^^fj^^^(\f^(^(v(srcy(sr(sr^c%rcsrr\rcsrc%r^i7ji75iTji7Ti7Ti7Ti77i7T^i7}^[TTi7ji7j^ 

cv (V fv fy fy (V fy fv fv fy fv fv (V fv fv fv fv (V (N 1 cv fv (V (V (V fv fy <v fv fy fv (V fv (V cv fv (V (V <v (V (V (V (V (V (V (V fv fv (V (V (V (V (V (V (V (V <v (V fy fy fy fy fy fy fyfyfyfy fy fy fy fyfy fy fy fy fy fy fyfy fyfy fy fy fyfyfyfyfy fy fy fy fy fyfy fy fyfyfyfy fyfyfyfy fy 
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Summary VRAM Reports 



A 

B 

C 

D 

1 

Name 

Total 

Texture 

Model 

2 

NatlReport 

139.9158 

89.4604 

50.4553 

3 

SkylReport 

137.6145 

89.9648 

47.6497 

4 

Sky3Report 

137.2639 

94.7876 

42.4763 

5 

IntrolReport 

136.4117 

85.4395 

50.9722 

6 

Quest 2Report 

134.2717 

108.437 

25.8347 

7 

Quest3Report 

130.0168 

100.3643 

29.6526 

8 

FinallReport 

129.9099 

86.0986 

43.8112 

9 

AcademyReport 

127.3535 

85.7944 

41.559 

10 

NatlReport 

126.1361 

79.3403 

46.7958 


Maps 
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Detailed VRAM Reports 



A 

B 

C 

D 

1 

File 

Total (MB) 

Texture 

Model 

53 

n o rrn a 1 m a p,t e xtu re s @ ■ 1 e v e Is ! n atl ! n at l_w e 1 co m e s ignO l_n tga, 110. igz 

0.375 

393216 

0 

54 

normalmap f textures@ 'levels lnatl!natl_woodenbucket_rVtga, 111. igz 

0.375 

393216 

0 

55 

norrnalrnap,textures@ [levels !natl!natl_woodpaintcliip01_n''tga. F lll,igz 

0.375 

393216 

0 

56 

norrmalmap^teKtures^ 1 'levels !questl!ql_v^oodtile01_n'tga. F lll,igz 

0.375 

393216 

0 

57 

n □ rm a 1 m a p,t e xtu re s @ Itre e s Lfl o w e r_catt a i 1 h ot d ogO l_n 'tga, 111. igz 

0.375 

393216 

0 

58 

Temporary/Bui Ids erver/xenon/Output/models/l eve Ia5sets/edc_prop5/5tatics/edc_tt_ho use low. igz 

0.3741 

0 

392232 

59 

TemporarY/BuildServer/xenon/Output/models/levelaBsetB/natl/statics/natl_woodenbucket.igz 

0.3629 

0 

380504 

60 

Temporary/BuildServer/xenon/Output/loosetexture5/cookies/caustic5tYle32half.igz 

0.3516 

368640 

0 

61 

Temporary/BuildServer/xenon/Output/model5/levelassets/edc_props/statics/edc_troubletakerhouse_flo\verset02.igz 

0.3513 

0 

368336 

62 

Temporary/BuildServer/Kenon/Output/actors/npc/flynn.igz 

0.3414 

0 

357976 

63 

colomnap f textures@'ui!resultscreen!star_land_dark'tga r 011.igz 

0.3359 

352256 

0 

64 

colormap,textures@iui!resultscreen!star_land_lit'tga f 011.igz 

0.3359 

352256 

0 

65 

colormap f teKtures@iui!resultscreen!star_sea_lit'tga. F 011.igz 

0.3359 

352256 

0 

66 

colornnap J .textures@iui!resultscreen!star_sea_unlit'tga,011.igz 

0.3359 

352256 

0 

67 

colQrmap,textures@'ui!resultscreen!star_skYjit'tga. F 011.igz 

0.3359 

352256 

0 

68 

co 1 o rm a p,t e xt u re s @ ■ u i ! res u It s ere e n ! st a r_s k y_u n 1 it'tga,011. igz 

0.3359 

352256 

0 


Maps 
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Package/Dependency Viewer 



cm material[ristances/proto215_sea_ping£phere_materials.nia1; (Xenon - charadters/AirLo.. 


Asset Name A. 

material lnstances/Airiock_Torpedo_Wartiead_materials. mat 

material lnstances/Airiock_Torpedo_Wartiead_Upgraded_materials .mat 

material Instances/Rn 1 ..materials .mat 

material Instances/Fin 2 _materials .mat 

material lnstances/FXM_TestSphere_Brian_materials .mat 

material lnstances/HolidayAiriock_H U D_materials .mat 


material lnstances/pnoto 21 5 _sea_ping Sphere_materials .mat 


material lnstances/UnderwaterMine_materials .mat 

material lnstances/UnderwaterMine_red_materials .mat 

material lnstances/UnderwaterMine_super_materials .mat 

models/Qiaractere/SkylanderE/Air 1 ock/air 1 ock_gun.igb 

models/'Characters/Skylanders/Airi0ck/airi0ck_gun_upgraded.igb 

models/tlharacters/Skylandere/Airiock/Airljackjjing Line .igb 

models/'Characters/Skylanders/Airiock/airioQkJorpedo.igb 

models/Eharacters/Skylanders/Airiock/airiockJorpedo^pgraded.igb 

models/Characters/SkylanderB/Airi0ck/airi0ck_t0rped0_warbead.igb 

models/Diaracters/Skylanders/Airi0ck/airi0ck_t0rped0_wartiead_upgraded.igb 

models/Characters/Skylanders/Airiock/Tin 1 .igb 

models/Characters/Skylanders/Airiock/'TinZ.igb 

models/Charactera/Skylanders/Airl0ck/FX_Airljock_PingOrde.igb 

mn dRjRiOismr^Rm/.SkvI^ndfim/Airinnk/FX Airl nnk Pinn.FinhRfiR inh 


* VRAM: 0/7319619 * Texture: 0/6281203 * Model: 0/1038416 * Audio: 0/2954102 i 


material lnstances/proto21 5_sea_ping Sphene_materials .mat 


1 ■■ materialinstances/pnotoZ 15_sea_pingsphene_materials .airlock _pingline,1 .igz 
(El- models/charactera/skylandem/airiock/airiock_pingline.igz 

Ej- vfx/characters/sk>ianders/airiock/airiock_sonarping_countermeasure.igz 

■ vfe/diaractera/skylandem/airiock/airiock_sonarping_helmet.igz 
■■■ vfx/charactem/skylandera/airiock/airiock_sonarping_helmet_upgrade01.igz 
]■ ^/charactem/skylandem/airiock/airiock_sonarping_netract.igz 

■ vtc. / charactem/skylandem/airiock/airiock_sonarping_netract_upgrade | [3 1 .igz 
+ ■■ materialinstances/pnofto2 1 5_sea_pingsphene_materials ,fx_airiock_pingcincle ,4.igz 
+j-materialinstances/pruto215_sea_pingsphene_materials 1 fx_airiock_pingsphene 1 2.igz 


m- 
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Performance 

• Baseline analysis 

• Typical gameplay 

• Reference character 

• Lead to budgets 

• Triangles, draw 
calls, vfx count, 
etc... 



CPU 
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Example Art Budgets 


iOS Low 
iOS High 


Gen7 


Draw 

Calls 

Triangles 

VFX 

Spawns 

300 

350,000 

150 

600 

700,000 

150 

1,500 

1,500,000 

150 

2,500 

2,500,000 

N/A 


Gen8 










Mr „■ 






Tools: Model Stats 


Draw Calls: 2346.43 / 2428.66 
Triangles: 2862683.75 / 2899863.66 
Vfx Count: 79.46 / 82.66 
igb count calls trls (skinned) 


sky3_ track 01 j v2 

1 

296 

848224 ( 

6) 

tree sharo02 

1 m 

26 

171641 ( 

0) 

s k y 3 _po rt a 1 d oo r _t01 

6 

72 

153672 ( 

0) 

rangeaoone 


7z 

J.Z//J.Z ( 

xz//xz) 

flynn 

1 

52 

114724 ( 114724) 

calicheap 

1 

44 

162884 ( 162884) 

s k y 3 _b omb ga t e01 

1 

25 

97665 ( 

97665 ) 

sky3_destructible_a 

11 

58 

93632 ( 

0) 
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Tools: Model Stats 



A 

B 

C D 

E 


F 

G 

H 

1 

model filename 

count igb draw calls gbuffer draw calls shadow draw calls 

total draw calls 

total tris 

gbuffertris shac 

2 

sky3_track01_v2 

1 

183 

60 

236 

296 

848224 

178048 

3 

tree_sharp02 

7 

2 

14 

12 

26 

171041 

92099 

4 

sky3_portaldoor_t01 

<c 

s\ 

48 

24 

72 

153672 

78492 

5 

rangedbone 

3 

9" 

27 

45 

72 

127712 

47892 

6 

flynn 

1 

13 

13 

39 

52 

114724 

28681 

7 

calicheap 

1 

11 

11 

33 

44 

102884 

25721 

S 

sky3_bombgate01 

1 

5 

5 

20 

25 

97065 

19413 

9 

s ky 3_d e stru cti b 1 e_a 

11 

2 

22 

36 

58 

93032 

35288 

10' 

sky3_island02 

16 

4 

48 

48 

112 

69280 

325 1 2 

11 

driverstealthelf 

1 

6 

6 

0 

24 

68404 

17101 

12 

free_sharp03 

1 

2 

2 

8 

10 

66025 

13205 

13 

s ky 3_d e st ru cti b 1 e_b 

21 

1 

21 

28 

49 

58016 

24864 

14 

free_sharp01 

3 

2 

6 

0 

6 

42027 

42027 

15 

sky3_tire01 

2 

6 

12 

12 

24 

41440 

20720 

16 

sky3_track01_flip 

1 

36 

0 

10 

10 

40784 

0 

17 

sky3_island04 

10 

6 

40 

24 

84 

39120 

21000 



ink 
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Normalized Meta Budgets 

• Approximate cost of "something" 

• Normalized to 100% per device class 

• Examples 

• Pixel cost 

• Entity 







Tools: Entity Cost 


,2016 #GDC16 
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Tools: Entity Cost 



A 


c 

D 

E 

F 

G 

H | 1 J IK L M N 

0 

p 

a 

1 1 

|Hash J 

[Type 

Namespat 

Name 

ID 

% Budget 

Enabled 

Enable State Report 




2 

5 

C Actor 

(null) 

Hero_driv 

40S7 

16,17 

1 

Enabled. 'Can Volume Cull 1 is false in data. Disable Volume Cull pushed by: Spawn Hero; Entity Attachrr 

3 

75 

CActor 

(null) 

AirJet; 

6125 

9.18 

1 

Enabled. 'Can Volume Cull 1 is false in data. Disable Volume Cull pushed by: Actor On Team Hero; Vehicl 

4 

3E+09 

CActor 

Magl_Lan 

FighterMs 

4175 

4.19 

1 

Enabled. 'Can Volume Cull 1 is false in data. Disable Volume Cull pushed by: Linear Vehicle Spline Movei 

5 

99449720 

CActor 

Magl_Libr Sharpfin j 

5018 

2.69 

1 

Enabled. 'Can Volume Cull' is false in data. 




6 

117 

CActor 

(null) 

encountei 

6727 

2.55 

1 

Enabled. 'Can Volume Cull' is false in data. 




7 

1.S4E+OS 

CGameEnl 

magljibr 

Collectibli 

5S07 

2.45 

1 

Enabled. Disable Volume Cull pushed by: Entity Attachment System. 




8 

109 

CActor 

(null) 

encountei 

6723 

2.41 

1 

Enabled. 'Can Volume Cull' is false in data. 




9 

9.07E+08 

CActor 

Magl_Libr 

Hugo_Cha 

4SS5 

2.28 

1 

Enabled. 'Can Volume Cull' is false in data. Disable Volume Cull pushed by: Entity Attachment System. 

10 

115 

CActor 

(null) 

encountei 

6726 

1.84 

1 

Enabled. 'Can Volume Cull' is false in data. 




11 

2.72E+09 

CPhysicalE Magl_Libr 

Pushblock 

5009 

1.8 

1 

Enabled. 'Can Volume Cull' is false in data. 




12 

119 

CActor 

(null) 

encountei 

6730 

1.66 

1 

Enabled. 'Can Volume Cull' is false in data. 




13 

6.44E+GS 

CActor 

MaglJJbr 

Maggs_Ch 

5002 

1.64 

1 

Enabled. 'Can Volume Cull 1 is false in data. 




14 

111 

CActor 

(null) 

encountei 

6724 

1.48 

1 

Enabled. 'Can Volume Cull' is false in data. 




15 

113 

CActor 

(null) 

encountei 

6725 

1.33 

1 

Enabled. 'Can Volume Cull' is false in data. 




16 

4.02E+G9 

CActor 

Magl_Libr 

FlynnS imp 

4SS3 

1.33 

1 

Enabled. 'Can Volume Cull' is false in data. 




17 

1.54E+09 

CActor 

Magl_Libr 

Cali_CharL 

4955 

1.15 

1 

Enabled. 'Can Volume Cull' is false in data. 




18 

423 2-S 

CPhysicalE 

runtime 

StraightSh 

6728 

0.52 

1 

Enabled. Disable Volume Cull pushed by: Projectile Component. 




IS 

S.34E+0S 

CGameEnl 

MaglJJbr 

DriverZon 

4980 

0.47 

1 

Enabled. 'Can Volume Cull' is false in data. 




20 

2.58 E+09 

CGameEnl 

Magl_Dm 

GiantRolle 

4741 

0.47 

1 

Enabled. 'Can Volume Cull' is false in data. 
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Level Performance Process 

• Data is recorded by QA 

• 2 consoles per day 

• Engineering processed data 

• Validates budgets and sends reports 

• QA writes issues 

• Developers fix issues 


File Edit Layout Connect * Utilities Help 


Search... 


tvi Vsc MOT <fFH chh bud dSl flN «Sfc Skylanders-2015 2857796-Main Q 


Performance Heat mapping 



behavior 

skyl 

aspen64 

cputime 

skyl 

aspen64 

draws 

skyl 

aspen64 

• droppedFramt skyl 

aspen64 

entity 

skyl 

aspen64 

gputime 

skyl 

aspen64 

gui 

skyl 

aspen64 

logic 

skyl 

aspen64 

other 

skyl 

aspen64 

physics 

skyl 

aspen64 

render 

skyl 

aspen64 

sound 

skyl 

aspen64 

totalTime 

skyl 

aspen64 

tris 

skyl 

aspen64 

vfxCount 

skyl 

aspen64 

vfxflush 

skyl 

aspen64 

vfx 

skyl 

aspen64 


Clear heatme 


LVL Outliner M Object Library [ 


Search Filter... 


3 

O # @ 

Object 

Type 

9 £ 

O * > 

■ Floor k 

Group 


o -• t> 

■i LandMasses 

Group 


o t> 

■ Mountains 

Group 


o « > 

■i Rocks 

Group 


O {> 

■ Stairs ^ 

Group 


« * 

ArtMountain223 

CStaticEntity 


o > 

ArtMountain224 

CStaticEnt'rty 


O 

Bridgefil 

CStaticEntity 


o » 

Design_Core_Platform_83 

CStaticEntity 


o 

Design_Core_Platform_137 

CStaticEntity 


o 

Design_Core_Platform_147 

CStaticEntity 
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Reports: Update Time 


Wiill Mean 1 Player Update [ms] 



Me an totarTinne 
l Mean cputime 
Me an gputime 
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Reports: Over Budget Frames 


WiiLI % Heatmap Points Over Budget 



■ Dropped Frames 

■ CPU Update 

■ GPU Update 
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Reports: Characters 


■ Max of drawCallCountRunningAverage ■ Average of drawCallCount 

Thrillipede 
Fiesta 

Turbo Charge Donkey Kong 
HammerSlam Bowser 
Nightfall 
Deep Dive Gill Grunt 
Dive-Clops 
Lava Lance Eruptor 
Bone Bash Roller Brawl 
Splat 

Shark Shooter Terraf in 
Hurricane Jet -Vac 
Big Bubble Pop Fizz 
SpitFire 

Double Dare Trigger Happy 
Sm ash H it 
High Volt 
Astroblast 
Super Shot Stealth Elf 
Storm Blade 

0 200 400 600 800 1000 1200 1400 1600 1800 2000 2200 2400 
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Splat 







Lava Lance Eruptor 











Dive-Clops 














Astroblast 







HammerSlam Bowser 
Bone Bash Roller Brawl 
Double Dare Trigger Happy 
SuperShot Stealth Elf 
Big Bubble Pop Fizz 



























Shark Shooter Terraf in 














Turbo Charge Donkey Kong 














SpitFire 














Hurricane Jet-Vac 







High Volt 
Storm Blade 
Smash Hit 


















Nightfall 














Deep Dive Gill Grunt 
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Reports: Trends 


Mean Total Update [ms] 
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Other Considerations 

• Context 

• High draw calls with low design log 

• Complex boss with few minions 

• Training and documentation 

• Platform Station 
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Talk Overview 

• Goals 

• Scalable Technology 

• Working with Budgets 

• Verification 

• Teams 

• Postmortem 
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Verification Cycle 

• Devs make changes 

• Build system creates 
and tests builds 

• QA tests builds and 
writes bugs 



Dev Policy for Scalability 

1. Test changes in game 

2. Trigger pre-submission testing (iomm) 

3. Submit 

4. Status set to "On the hook" 

5. Wait for dev build with changes (30 min) 

6. Status set to "Off the hook" 


Pre-submission testing 


• Basic testing 

• Compile 

• Verify memory 

• Bug management 

• Reviews 

• Database integration 
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Pre-submission testing 


• Advanced Testing 

• More devices 

• Memory validation 
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Build Generation 

• Main build 

• Basic validation 

• Device builds 

• Testing 

• Builds for QA, 
Marketing, Kid 
testing, etc... 


:: pulse 2.7 :: 

browse server agents administration 


o 

a Supernova 



Q 

a Nova Project Template 

8 projects building 



skylanders-main 

R build 5777 0 

:: V" 

9 

supernova-code-analysis 

0 build 19260 0 

" 


supernova-code-aspen 

0 build 110099 0 

:: © success 

• |( 

supernova-copy-alchemy.tool-to-automatedrelease 0 this project has never been built 

a 

supernova-copy-alchemy.tool-to- release 

0 this project has never been built 

a 

supernova-copy-alchemy. tool-to-releasecandidate 

0 this project has never been built 

o 

sb pemova-data-aspen 

0 build | 2206 0 

:: 

o 

supemova-live-ops-main 

0 build 279 0 

:: ^ warnings 






supernova-localization 

0 build 385 0 

:: ^ warnings 





9 

super nova -long 

- build 10905 0 


9 

supernova-long-cafe 

0 build 16022 0 

:: 0 failure 

O 

supernova-long-durango 

0 build 9797 0 

:: ^ warnings 

°a 

su pernova - long- ps3 

0 build 12059 0 

:: ^ warnings 



O 

supernova-long-ps4 

0 build 6790 0 


O 

supernova-long-win 

0 build 13630 0 

:: 0 failure 


supernova-main 

0 build -31357 [0 
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Automated Testing 

. APIs 

. I/O, Memory, VFX 

• Features 

• Rendering, race mode 

• Content 

• Levels, characters, 
vehicles 
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Hardware 


• 41 pre-submission machines 

• 31 build machines 

• 38 devkits 

. 3 PS3, 3 PS4, 3 WiiU, 2 XB1, 25 XB360, 2 
iPad 
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• Local [30 PPL] 

• Developer support 

• Performance 

• SKU Parity 

• Build Verification Testing (BVTs) 

• External [~130 PPL] 

• Additional testing and 1 st Party Support 
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Build Verification Tests (BVTs) 


• Consistent testing 

• Examples 

•Level progression 
•Character upgrades 
•Collectibles 
•Hats 

• Reports to Production 


Level 

Status 

Notes 

BLOCKER 
Bug ID 

OTHER 
Bug ID 

Intro 

PASS 


- 

- 

Intro 

1 

PASS 


- 

- 

Sky 1 

PASS 


- 

- 

Sky 2 

PASS 


- 

- 

Sky 3 

PASS 


- 

- 

Quest 

1 

PASS 

CRITICAL - 

Occasional 

crash 

during 

fights 

- 

SNV- 

49141 

SNV- 

52128 

Magic 

1 

FAIL 

BLOCKER: 
Collision 
misaligned 
or non- 
existent on 
level 

geometry 

SNV-71079 

- 

Natl 

PASS 


- 

- 
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GDC 


Technology for QA 


• Build retrieval 

• Easy deployment 

• Efficiency gains 

• Add gold 

• XP multiplier 

• Skip movies 



QftTool Bug 


Smoke 

Options 

/y pbl'clpliar ^ 

Cha racier 

KidTestMode 

Vehicle 

Allow Skipping Cuirscenes 


Show Toy Info 
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Talk Overview 

• Goals 

• Scalable Technology 

• Working with Budgets 

• Verification 

• Teams 

• Postmortem 
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Device Agnostic [80% PPL] 

• Content focused teams p-ioppl] 

• Characters, vehicles, levels, etc. 

• Work with budgets 

• No device hardware 
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Cross-device 

• UX team [3-5 ppl] 

• 1 st Party Requirements 

• Feature teams [5-7 ppl] 

• Online & Rendering 



• Optimization team [sppl] 

• Memory/performance 
optimization and management 




i« : * 
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Device Specific 

• iOS team [3-5 ppu 

• Custom UI 

• In app purchasing 

• Touch controls 

• Streaming assets 
from cloud 
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Support 

• Build team [2-3 PPL] 

• Build maintenance 

• Build delivery 

• Developer support 




ink 
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Talk Overview 

• Goals 

• Scalable Technology 

• Working with Budgets 

• Verification 

• Teams 

• Postmortem 
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What worked? 

• Platform emulation 

• Most developers ignored devices 

• Budgets 

• Content creators understood scalability 
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What's next? 

• Less device classes 

• Early budgets 

• More importance authoring 

• Scripting 

• More procedural systems 

. VFX 
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What's next? 

• Offline memory analysis 

• Faster submission process 

• More automated testing 



